Supporting Efficient Noncontiguous Access in PVFS over InfiniBand

نویسندگان

  • Jiesheng Wu
  • Pete Wyckoff
  • Dhabaleswar K. Panda
چکیده

Noncontiguous I/O access is the main access pattern in many scientific applications. Noncontiguity exists both in access to files and in access to target memory regions on the client. This characteristic imposes a requirement of native noncontiguous I/O access support in cluster file systems for high performance. In this paper, we address two main issues on supporting efficient noncontiguous I/O access in cluster file systems over a high performance network. One is noncontiguous data transmission between the client and the I/O server. The second is noncontiguous disk access on the I/O server itself. We propose a novel approach, RDMA Gather/Scatter, to transfer noncontiguous data for such I/O accesses, and design a new scheme, Optimistic Group Registration, to reduce memory registration costs associated with this approach. For the second issue, we deploy data sieving on the I/O server to process a large number of small noncontiguous disk accesses. Unlike other data sieving implementations, our I/O server uses a cost model actively and intelligently to decide whether it is beneficial to perform data sieving or not. We have designed and incorporated these approaches in a version of PVFS over InfiniBand. Through a range of PVFS and MPI-IO micro-benchmarks, the MPI-IO tiled access test and the NAS BTIO benchmark, we demonstrate that our approaches attain significant performance gains compared to other existing approaches.

برای دانلود رایگان متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Benefits of Quadrics Scatter / Gather to PVFS 2

Noncontiguous IO access is the main access pattern in scientific applications. We have designed an algorithm that supports zerocopy noncontiguous PVFS2 IO using a software scatter/gather mechanism over Quadrics. To investigate what impact Quadrics scatter/gather mechanism can have on PVFS2 IO accesses, in this paper, we perform an indepth evaluation of the scatter/gather mechanism. We also stud...

متن کامل

ts of Quadrics Scatter / Gather to PVFS 2

Noncontiguous IO access is the main access pattern in scientific applications. We have designed an algorithm that supports zerocopy noncontiguous PVFS2 IO using a software scatter/gather mechanism over Quadrics. To investigate what impact Quadrics scatter/gather mechanism can have on PVFS2 IO accesses, in this paper, we perform an indepth evaluation of the scatter/gather mechanism. We also stud...

متن کامل

PVFS over InfiniBand: Design and Performance Evaluation

I/O is quickly emerging as the main bottleneck limiting performance in modern day clusters. The need for scalable parallel I/O and file systems is becoming more and more urgent. In this paper, we examine the feasibility of leveraging InfiniBand technology to improve I/O performance and scalability of cluster file systems. We use Parallel Virtual File System (PVFS) as a basis for exploring these...

متن کامل

Evaluating structured I/O methods for parallel file systems

Modern data-intensive structured datasets constantly undergo manipulation and migration through parallel scientific applications. Directly supporting these time consuming operations is an important step in providing high-performance I/O solutions for modern large-scale applications. High-level interfaces such as HDF5 and Parallel netCDF provide convenient APIs for accessing structured datasets,...

متن کامل

Noncontiguous I/O Accesses Through MPI-IO

I/O performance remains a weakness of parallel computing systems today. While this weakness is partly attributed to rapid advances in other system components, I/O interfaces available to programmers and the I/O methods supported by file systems have traditionally not matched efficiently with the types of I/O operations that scientific applications perform, particularly noncontiguous accesses. T...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

عنوان ژورنال:

دوره   شماره 

صفحات  -

تاریخ انتشار 2003